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Abstract 

The validity of speech act data taken from different kinds of elicitation instruments has been widely discussed in 
interlangauge and cross cultural pragmatics literature. In this study an attempt is made to evaluate and investigate data 
taken from two of the most popular speech act instruments namely, written DCT and closed role play. The responses of 
forty Iranian university students in their native language (Persian) were investigated across the speech act of request. 
The distinguishing feature of the study is using the same participants for the two methods; in this way we can have an 
account of intra-participant variations. The social status and distance of the participants were both equal. Differences 
were found in the length and content of the responses. Respondents to role play tended to have longer responses and it 
was mainly because of the longer and higher number of alerters and supportive moves used in the role play. In the 
written data we witnessed more direct strategies used in the request Head Acts. Likewise, modification devices used in 
the oral data had a softer tone and in terms of request perspective the oral data provided more impersonal responses 
while the requests in the written data were more hearer-oriented. Overall, it seems that the data gathered through role 
play is more natural than DCT. Finally it is discussed that the choice of data gathering instrument in pragmatics research 
is to a large extent dependent on the goals of the researcher and research questions. 

Keywords: Speech act instrument. Intra-participant variation. Social status. Social distance 

1. Introduction 

Researchers in the field of pragmatic studies have emphasized the importance of methodological issues (Kasper & Dahl, 
1991; Cohen, 1996). The choice of the instrument will determine the reliability and accuracy of the data that represent 
the authentic performance of the language related behavior. It has been mentioned that discourse completion task (DCT) 
and role play are two primary elicitation measures used in speech act and cross cultural pragmatic studies (kasper& Dahl, 
1991; Chaudron, 2005). These two tasks are especially used in EFL settings since it is difficult to collect sufficient 
naturally occurring data in these settings and also because of the problem of controlling the pragmatic parameters in 
naturally-occuring conversation. The validity of these two tasks has become especially important for researchers. No 
matter what else we do, we must remember that if data are inadequate, there is always the danger that the theory and 
conclusion drawn from them could be unreliable and misleading (Wolfsen, 1986). 

DCTs put forth a situation in which respondents are expected to produce what they deem to be appropriate for that 
particular situation. DCTs have been widely used in speech act studies because of the several advantages. They let the 
researcher collect a large amount of data quickly (Yamashita, 1996). Also they make it possible to control situational 
variables such as social status, age, gender, and L2 proficiency (Kasper, 2000). Another advantage of DCTs is that there 
is no need for transcription which can cause easier comparison of the data elicited from different respondents (Chaudron, 
2005). DCTs can also be administered to a large number of respondents and therefore making it possible to compare the 
responses of native and nonnative speakers across different cultures (Blum-Kulka et ah, 1989a). 

But DCTs are problematic when it comes to validity. (Golato, 2003). DCTs are decontextualized and respondents do not face 
the real life situations so the data represents what people think they would say not what they actually say (Aston, 2004). 
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Another shortcoming related to validity is that respondents' oral performance is intended to be measured through the written 
mode (Kasper & Dahl. 1991). Completers of DCTs may also think of the written task as some formal activity and as a result 
may apply more formal language (Beebe & Cummings, 1996). 

In role play participants are asked to take a particular role requiring the completion of a speech act. Role plays have been 
divided into two major categories. In case, the participants are given freedom to interact with the interlocutor we have 
open role plays but when the respondent is given few or no opportunities to interact we have closed role plays. Open role 
plays are more time consuming to conduct since they take more conversational turns. In closed role plays it is easier to 
control the procedure. Role play has been used widely for deriving behaviors of native and nonnative speakers in a 
variety of speech acts such as apologies, requests, refusals, compliments, and gratitude since it solves some of the 
shortcomings of DCTs. Data gathered through role plays is in the oral mode rather than written mode and this makes the 
data closer to what and as much as what respondents would say in real situations (Houck &Gass, 1996). Also features of 
natural discourse such as turn-taking, false starts, and hesitation reflect natural speeches in role plays (Kasper & Dahl, 
1991). Nevertheless, the data gathered through role plays has come under question. In role plays, participants are not 
involved in naturally occurring interactions and are just playing the roles in specific situations and this causes the 
responses to be uncertain reflections of real language behavior (Kasper, 2000). 

A number of studies have juxtaposed the data taken from of DCTs and role plays. Reintell and Mitchel (1989) compared 
closed role play and DCT for eliciting requests and apologies from native speakers and non-native speakers in terms of 
response length and range of linguistic strategies. They witnessed that in role plays non native speaker respondents 
produced much longer responses than DCTs. Reportedly, this length difference was due to non-native speakers' longer 
supportive moves, repetitions, and hesitations. Also the oral data tended to be more direct than the written mode. Beebe 
et al., (1996) and Wolfson et al., (1989) found a number of differences between data collected by DCT and role play In 
terms of actual wording used in real interpersonal interaction, the length of response or the number of turns it takes to 
fulfill the function, the number of repetitions and elaborations that occur, and the depth of emotion that in turn 
qualitatively affects the tone, content, and the form of linguistic performance. 

It can be mentioned that the previous research on comparing these two tasks has a number of limitations. First, the data 
gathered is not that much comparable because they were elicited from different participants with diverse backgrounds in 
mother tongue, age, education, and cultural background as these variables may affect the result (Yamashita, 1996). Using 
the same participants for both methods is important when checking the validity and reliability of these instruments as 
evaluation measures (Alderson et al., 1995). Second, situations used for DCTs and role plays were often not appropriate 
for all types of respondents. In many of these studies students were put in situations that seem inappropriate. For 
example in Hudson et al. (1995) study students have to take the role of a person who holds a lease from the bank or an 
individual playing the role of a manager hiring employees. 

The study tries to compare participants' responses (native language) elicited from DCTs and role plays with regards to 
length and content; in doing so the same respondents have been used. The participants are native speakers of Persian. In 
interlangauge and cross cultural studies usually the students' performance in their native language is compared to their 
performance in the second language or performance of the native speakers of language A is compared to performance of 
native speakers of language B or sometimes the answers of native speakers of a language are evaluated against responses 
of speakers of a second language. With this in mind, it seemed necessary for the researchers to investigate whether 
performance of native speakers of Persian measured by the two mentioned instruments are the same. 

2. Methodology 

2.1 Participants 

Forty university male students majoring in Mechanical engineering were chosen for this study. They were between the 
age ranges of 18 to 22. 

2.2 Instruments 

Our data were gathered by means of role play and DCT. Eight request situations were devised for both the role play and 
DCT. On DCT, respondents were provided with given situations which required them to write down responses they 
believed are most appropriate for that context. We made use of closed role plays in which participants are not allowed to 
interact freely with the interlocutor. Respondents were asked to take the assumed role and pretend that their responses 
are part of longer conversation therefore some degree of unnaturalness was imposed on our participants. 

All the participants in our study were university students and in this way it was made possible to control their 
educational background and age. The roles respondents played were as natural as possible since this role was part of 
their life. Participants would not perform a role which was different from them e.g., a boss, a pilot, or a professor. 
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Trosborg (1995) believes that imposing roles which respondents are not familiar with would impact the naturalness of 
the performance. In our scenarios both the social status and the social distance was equal; the situations for the study 
were those that would occur between classmates or friends in their university setting or daily life. 

2.3 Procedure 

The forty participants were first given the handout in which the situations were explained; they were given enough time to 
study the situations. Then two by two they were taken to another room to perform the role plays and were audio-recorded. 
After they finished the eight situations, they were released and the next pair started the procedure. So here we had twenty requests. 

Ten days after the role plays, the twenty students who had made requests were asked to complete a DCT. The completers 
were provided with exactly the same situations they were exposed to for role play. Respondents had enough time and 
space to complete the DCT. The idea for administering the role play before the DCT was to give spontaneity and to 
reduce the practice effect of the role play. 

2.4 Data Analysis 

The data was assessed with regards to length and content of the responses elicited from role plays and DCTs. Regarding role 
plays, all the conversations were transcribed. The parts which were not relevant to the speech act of request were omitted. For 
instance participants" statement after the request was conceived by the interlocutor was not analyzed. To examine the length of 
requests made, repetition or hesitation was counted as one word and interjections were not counted. 

To investigate requests. Cross Cultural Study of Speech Act Realization Patterns (CCASRP) coding scheme was applied 
(see the Appendix of Blum-Kulka et al., 1989a). This design indicates that requests are comprised of the following parts: 
Head Act (e.g., ‘could you do the cleaning up?’), alerter (e.g.,’ excuse me’, ‘Joe’), preposed supportive moves (e.g., ‘I 
really need a favor’), postposed supportive move (e.g., ‘I promise I'll have it ready by tomorrow’). In this 
categorizations, nine strategies have been considered for Head Acts based on the degree of directness. The strategies are: 

1- Mood derivable: e.g.. Clean the table. 

2- Explicit performative: e.g., I am asking you to clean the table. 

3- Hedged performative: e.g., I have to ask you to clean the table. 

4- Locution derivable: e.g.. You have to clean the table. 

5- Want statement: e.g., I want the table cleaned. 

6- Suggestory formula: e.g.. How about cleaning the table? 

7- Prepatory: e.g.. Could you clan the table? 

8- Strong hint: e.g.. The table is dirty. 

9- Mild hint: e.g., I am exhausted to clean the table. 

Also the Head Act can be modified by devices such as lexical/phrasal downgrades (marker please, consultive devices, 
downtoners, understaters/hedges, subjectivisers, cajolers, appealers) or syntactic downgraders (conditional structures, 
conditional clause, tense, aspect, interrogative, negation of prepatory condition). The preposed and postposed supportive 
moves can also be modified by devices such as grounder, disarmer, prepator, getting a precommitment, promise, 
imposition minimiser, apology, and discourse organization move. 

Request perspective was another variable which was investigated in the two types of data. Requests usually include 
reference to the requester, the recipient of the request, and/or the action to be performed. The speaker can manipulate 
requests by choosing from a variety of perspectives (Blum-Kulka et al., 1989b) in making requests: Hearer-oriented 
(emphasis on the role of the hearer). Speaker-oriented (emphasis on the speaker’s role as the requester). Speaker- and 
hearer-oriented (inclusive strategy), and Impersonal. 

Therefore, the students' responses to DCT (written) and role play (oral) were compared in terms of length of alerters, 
supportive moves. Head Acts, and choice of perspective. 

3. Result and Discussion 

3.1. Response Length 

Our analyses show a significant difference between the written and oral data in terms of length of responses. As Table 1 
indicates the role play responses are significantly longer than DCT answers although they are produced by the same 
completers. 

<Table 1 about hero 
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This mean difference can be explained by the nature of role plays which cause more hesitation, inversion, and repetition. 
Another reason for this length difference was the use of more and longer alerters and supportive moves used by role play 
participants. As an example in oral data the participants used alerters such as ‘Bebakhshid’ ‘Excuse me’, 
‘Ozrmikhamghorban’ ‘Excuse me sir’ much more than the written data. This finding confirms (Blum-Kulka et al., 1989b) 
who highlighted the importance of alerters in conversations. Also the study by Yuan (2001) found a significant 
difference between the length of responses in the oral and written data but since here the same participants were used in 
both mediums our result is more solid. Based on the above, we can state that role play generates more natural data than 
DCT. 

3.2. Alerters and Supportive Moves 

As Table 2 indicates there is a significant difference between frequency of alerters in the oral and written data. In the role 
play we have a higher number of both alerters and supportive moves than DCT. These alerters are used in role play to 
draw the attention of the audience but this does not happen in the written data. Regarding supportive moves we can 
claim that since the directions (prompts) in the DCT cannot instigate the actual situation, respondents use shorter or no 
supportive moves. It seems that role play is more effective in eliciting what completers would do in real interaction. 

<Table 2 about hero 

In terms of type of alerters, completers used more types of alerters in role play than in DCT. In Role play four types of 
alerters were used while in DCT two types were elicited. The most frequent alerter used in both DCT and role play was 
‘bebakhshid, Ozrmikham’, ‘Excuse me’ (Table 3). 

<Tables3 & 4 about hero 

As table 4 indicates for supportive moves. Role play data show six different devices but in DCT responses only four 
supportive moves were identified. Grounder was the most frequent supportive move in both DCT and role play followed 
by disarmer, prepator, and imposition minimizer. Only in role play apology and discourse organization move were used. 
This shows that respondents can better feel the interactive nature of real situations in role plays. 

3.3 Head Act Strategies 

AS Table 5 shows some strategies are used more often in the oral data than the written data. What can be seen is that 
DCT responses include more direct strategies than those of the role play. Other studies (e.g., Blum-Kulka et al., 
1989a ;Reintel& Mitchel, 1989) that investigated the behavior of native English speakers also indicate the same pattern. 
It seems that there are factors in role plays which cause respondents to be less direct. This intra-participant variation 
indicates that different methods can elicit different data from the same respondents. So if different completers were 
measured through different methods, we could not have an account of these variations. 

Table 5 indicates that strategies 4 & 9 were not used in either method. Strategy 2 was used in DCT but not in Role play 
and strategy 8 was used in role play but not in DCT. So Head act strategies are different in these two data elicitation 
methods. 

<Table 5 about hero 

3.4 Internal Modifications Used in the Head Acts 

In general we encountered a high number of lexical and syntactic downgrades and the reason can be high level of 
proficiency of participants in their mother tongue. Takahashi and Beebe (1987) believe that participants with a good 
level of language proficiency elaborate speech act expressions. Lexical downgrades such as subjectivisers and appealers 
were not used in both written and oral data. The only syntactic downgrader not used in either DCT and role play was 
negation of prepatory condition. Table 6 indicates that Role play data reflect a higher number of downgraders than the 
DCT data (both syntactic and lexical). It indicates that in conversation respondents may have opted to apply a milder 
tone than in DCT. 


<Table 6 about hero 

3.5 Request Perspective 

As Table 7 shows respondents to role play had the tendency to use more joint perspective and impersonal perspective 
while DCT completers' replies were more Hearer dominant. It indicates that in role play respondents are more cautious 
which can be considered as more natural. 

<Table 7 about hero 
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4. Conclusion 

Pragmatic competence is at the heart of current trends of language pedagogy. It is believed that failure in the mastery of 
speech act will cause communicative breakdown (Wolfson, 1989). Therefore, it seems necessary to have research in the 
issue of data collection method in order to confirm the validity and reliability of the instruments of data collection. 

The study compares the data elicited from DCT and role play across the speech act of request. We used the same 
participants for both groups with the aim of having a controlled methodological comparison. In the data from the two 
instruments differences in terms of length and content of responses were found. It has been shown that completers 
provide longer responses in role play which is mostly because of the higher number of alerters and supportive moves 
used in the oral data. There was no difference between the varieties of strategies used in the two methods but in the 
written data more direct strategies were applied. Modification devices used in role play had a softer tone and finally the 
written data were more hearer-oriented while the oral data were more impersonal or collective referent in terms of 
perspective. Based on our findings we can state that role play is a more appropriate technique than DCT if we consider 
naturalness of speech. Turnbull (1997) outlines four criteria for good pragmatic elicitation technique; He argues that role 
play provides data which is representative of natural speech, it is efficient with regards to the extent of data it can gather, 
it gives room for researcher control, and finally it is ethical. 

It should be mentioned that goals of the researcher and the research questions to a large extent determine the choice of 
speech act data elicitation instrument. For example. Although, DCT is not without its limitations, it can be used in 
investigation of a given speech act in a particular language in an early stage in order to delineate comprehension patterns. 
The findings of this study are grounded on the particular speech act of requesting by Persian speakers elicited from 
written DCT and closed role play in a testing situation while the social status and distance between respondents were 
equal. It is beyond denial that the use of different data collection techniques in different research issues needs empirical 
inquiry. However the conclusions drawn from this study should make researchers aware of not delving into the issue of 
pragmatic competence based on a single data collection instrument. 
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Table 1. Mean response length of the participants 


Instrument 

Mean 

df 

SD 

DCT 

130.55 

19 

0.65* 

Role play 

144.44 

19 

0.52* 


* P < 0.01 


Table 2. Mean frequency of alerters and supportive moves in participants' requests 


Formula 

Mean Frequency 

p-value (Chi square) 

DCT 

Role play 

Alerter 

15.2 

24.4 

.00 

Supportive move 

26.3 

32.5 

.00 


Table 3. Mean number and percentage of alerters used in participants' responses 


Type 

DCT 

Role play 

Number 

Percentage 

Number 

Percentage 

Bebakhshid, Ozrmikham 

9 

60% 

11 


45/8% 

Excuse me 






Salam 

6 

40% 

4 


16/6% 

Hello 






Khastehnabashid 

0 

0 

5 


20/8% 

Cheer up 






First name (e.g., Ali) 

0 

0 

4 

16/6 
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Table 4. Mean number and percentage of supportive moves used in participants' responses 


Type 

DCT 

Role play 

Number 

Percentage 

Number 

Percentage 

Grounder 

10 

38.4 

8 

25 

Disarmer 

8 

30.7 

7 

21.8 

prepator 

5 

19.2 

5 

15.6 

Imposition minimizer 

3 

11.5 

5 

15.6 

Apology 

0 

0 

5 

15.6 

Discourse organization move 

0 

0 

2 

6.2 


Table 5. Types of Strategies Used in the Head Acts 


Category 

DCT 

Role play 

1 . 

Mood derivable 

45% 

26% 

2. 

Explicit performative 

15% 

0 

3. 

Heded performative 

15% 

9% 

4. 

Locution derivable 

0 

0 

5. 

Want statement 

8% 

19% 

6. 

Suggestory formula 

15% 

11% 

7. 

Preparatory 

12% 

21% 

8. 

Strong hint 

0 

14% 

9. 

Mild hint 

0 

0 


Table 6. Types of internal modifications used in the Head Acts 


Modifier 

DCT 

Role play 

Lexical/phrasal downgrades 

Marker ‘please’, consultive devices, downtoners, hedges, 
cajoler,. 

35% 

65% 

Syntactic downgraders 

Conditional structures, conditional clauses, tense, aspect, 
interogatives 

41% 

59% 


Table 7. Choice of perspective by respondents in DCT and Role play 


Perspective 

DCT 

Role play 

Hearer dominance 

41% 

14% 

Speaker dominance 

31% 

27% 

Joint perspective 

18% 

39% 

Impersonal 

10% 

20% 
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